In a mixed reality system to present, to an observer, a known mixed reality space in which a physical space and a virtual space are blended, a position/orientation measuring device represented by FASTRAK (trademark) available from Polhemus (U.S.A.) is generally used to acquire the viewpoint position and the direction of line of sight in observing the mixed reality space (e.g., U.S. Pat. No. 6,522,312).
As an image sensing/display device, an HMD (Head Mounted Display) with a camera is often used. Stereoscopic vision by binocular parallax is also implemented by separately providing left and right image sensing systems and display systems in an HMD.
For the above-described mixed reality system, the present inventor has proposed a technique of laying out CAD data in a virtual space as a virtual object in the virtual space, generating an image obtained by viewing the virtual object from the viewpoint position and the direction of line of sight, and displaying the image on the display device provided on the HMD. When a virtual object corresponding to virtual CAD data is displayed in a physical space image, the viewpoint can easily and freely be moved. In addition, since the virtual object is composited with the physical space image, the sense of scale for the virtual object can be obtained, and more real display can be executed.
On the other hand, a conventional CAD system which uses no mixed reality system has an important function of displaying an assembly tree. Each part in a design target object is selected from an assembly tree and manipulated. When the assembly tree is used, a common operation can be executed at once for a plurality of integrated components.
However, in the mixed reality system, generally, the physical space image is displayed on the display device provided on the HMD. Hence, the display region to display various kinds of information is small. This also results from the small display area of the display device of the HMD.
When stereoscopic vision is implemented by using a stereo HMD, and an assembly tree is carelessly displayed, a sense of discomfort is generated in the stereoscopic vision.